System testing is generally used to test a product such as a software application in an integrated system. A system can include software, hardware, and the like. Current methods for system testing of a product begin with building a test plan. The test plan provides a sparse coverage matrix of a particular product based on its documented design and customer usage cases. The documented design and customer usage cases are generated prior to the product's creation. The system test plan is one of the measurement devices that is used to determine if the quality and function of the product has been verified. After the product is created it can be system tested to identify defects and behavior anomalies, which are reported by the testers. Usually a subset of these defects and anomalies are fixed within the current release cycle.
One problem with the current state of system testing is that the system test plan is created prior the product being created. Therefore, the abilities of any particular development team at any given point in time cannot be taken into account. In any given development team at any particular point in time (which is a particular release cycle of the product) there are certain abilities and weaknesses. For example, contractors may have little experience with deployment tools, the product design, environment and user cases for the product, and the like. The technology being deployed can be new to the team members. The design process can also change during the design of the product or the product implementation can change. These are only a few examples of events that cannot be taken into account if a system test plan is created prior to the product's creation.
Accordingly, a design based system test plan is not an adequate method to discover defects in a product. Design based plans focus on what was thought to be an issue in design, which may not be an issue at all. This causes test resources to be wasted on non-problematic areas leaving other areas not adequately tested.
Therefore a need exists to overcome the problems with the prior art as discussed above.